Gaming controller

ABSTRACT

A controller for interfacing wirelessly with a computing device is provided, including the following: a housing defined by a main body, a first extension extending from a first end of the main body, and a second extension extending from a second end of the main body, the first extension and the second extension for holding by a first hand and a second hand of a user, respectively; a touchscreen defined along the top surface of the main body between the first extension and the second extension; a first set of buttons disposed on the top surface of the main body proximate to the first extension and on a first side of the touchscreen; and a second set of buttons disposed on the top surface of the main body proximate to the second extension and on a second side of the touchscreen.

CLAIM OF PRIORITY

This application claims priority as a continuation of U.S. applicationSer. No. 15/133,173, filed Apr. 19, 2016, entitled “Gaming Controller,”which is a continuation of U.S. application Ser. No. 14/752,808 (nowU.S. Pat. No. 9,381,435), filed Jun. 26, 2015, entitled “GamingController,” which is a continuation of U.S. application Ser. No.13/842,975 (now U.S. Pat. No. 9,116,555), filed Mar. 15, 2013, entitled“Gaming Controller,” which is a continuation-in-part of U.S. applicationSer. No. 13/539,311 (now U.S. Pat. No. 8,870,654), filed Jun. 30, 2012,entitled “Gaming Controller,” which claims priority to U.S. ProvisionalPatent Application No. 61/563,505, filed Nov. 23, 2011.

U.S. application Ser. No. 13/842,975 further claims priority to U.S.Provisional Application No. 61/745,281, filed Dec. 21, 2012, entitled“Automatic Generation of Suggested Mini-Games for Cloud-Gaming Based onRecorded Gameplay.”

U.S. application Ser. No. 13/842,975 further claims priority as acontinuation-in-part of U.S. application Ser. No. 13/418,691 (now U.S.Pat. No. 8,672,765) filed Mar. 13, 2012, entitled “System and Method forCapturing and Sharing Console Gaming Data.”

U.S. application Ser. No. 13/842,975 further claims priority as acontinuation-in-part of U.S. application Ser. No. 13/418,703, filed Mar.13, 2012, entitled “System and Method for Capturing and Sharing ConsoleGaming Data.”

U.S. application Ser. No. 13/842,975 further claims priority as acontinuation-in-part of U.S. application Ser. No. 13/418,720, filed Mar.13, 2012, entitled “System and Method for Capturing and Sharing ConsoleGaming Data.”

The disclosures of these applications are incorporated by referenceherein in their entirety for all purposes.

RELATED APPLICATIONS

This application is related to U.S. application Ser. No. 13/839,382filed Mar. 15, 2013, entitled “Sharing Recorded Gameplay to a SocialGraph,” the disclosure of which is incorporated by reference herein.

BACKGROUND 1. Field of the Invention

The present invention relates to controllers for interfacing with aninteractive program.

2. Description of the Related Art

The video game industry has seen many changes over the years. Ascomputing power has expanded, developers of video games have likewisecreated game software that takes advantage of these increases incomputing power. To this end, video game developers have been codinggames that incorporate sophisticated operations and mathematics toproduce a very realistic game experience.

Example gaming platforms, may be the Sony Playstation®, SonyPlaystation2® (PS2), and Sony Playstation3® (PS3), each of which is soldin the form of a game console. As is well known, the game console isdesigned to connect to a monitor (usually a television) and enable userinteraction through handheld controllers. One example of a handheldcontroller is the DUALSHOCK® 3 wireless controller manufactured by SonyComputer Entertainment Inc.

It is in this context that embodiments of the invention arise.

SUMMARY

Embodiments of the present invention provide a controller forinterfacing with an interactive application such as a video game.Several inventive embodiments of the present invention are describedbelow.

In one embodiment, a controller for interfacing with an interactiveapplication is provided, including: a housing defined by a main body, afirst extension extending from a first end of the main body, and asecond extension extending from a second end of the main body, the firstextension and the second extension for holding by a first hand and asecond hand of a user, respectively; an input device positioned along atop surface of the main body; a touch-sensitive panel defined along thetop surface of the main body.

In one embodiment, the input device is selected from the groupconsisting of a joystick, a button, a trigger, a directional pad.

In one embodiment, the controller further includes a tracking paneldefined along a front side surface of the main body; and a light definedin the main body for illuminating the tracking panel.

In one embodiment, the tracking panel is defined from a translucentmaterial.

In one embodiment, the controller further includes one or more of anaccelerometer, a gyroscope, or a magnetometer.

In one embodiment, the input device is a button, the button beingconfigured to activate a sharing interface for sharing recorded gameplayto a social graph of a user.

In one embodiment, sharing recorded gameplay includes sharing one ormore of an image or a video clip.

In one embodiment, sharing recorded gameplay includes streaming a livevideo feed of a user's gameplay.

In another embodiment, a controller for interfacing with an interactiveapplication includes: a housing defined by a main body, a firstextension extending from a first end of the main body, and a secondextension extending from a second end of the main body, the firstextension and the second extension for holding by a first hand and asecond hand of a user, respectively; a button positioned along a topsurface of the main body, the button being configured to activate asharing interface for sharing recorded gameplay to a social graph of auser.

In one embodiment, sharing recorded gameplay includes sharing one ormore of an image or a video clip.

In one embodiment, sharing recorded gameplay includes streaming a livevideo feed of a user's gameplay.

In one embodiment, the controller further includes a touch-sensitivepanel defined along the top surface of the main body.

In one embodiment, the controller further includes an input devicepositioned along a top surface of the main body.

In one embodiment, the input device is selected from the groupconsisting of a joystick, a button, a trigger, a directional pad.

In one embodiment, the controller further includes a tracking paneldefined along a front side surface of the main body; and a light definedin the main body for illuminating the tracking panel.

In one embodiment, the tracking panel is defined from a translucentmaterial.

In one embodiment, the controller further includes one or more of anaccelerometer, a gyroscope, or a magnetometer.

In another embodiment, a controller for interfacing with an interactiveapplication is provided, including: a housing defined by a main body, afirst extension extending from a first end of the main body, and asecond extension extending from a second end of the main body, the firstextension and the second extension for holding by a first hand and asecond hand of a user, respectively; a touch-sensitive panel definedalong the top surface of the main body; a button positioned along a topsurface of the main body, the button being configured to activate asharing interface for sharing recorded gameplay to a social graph of auser; a tracking panel defined along a front side surface of the mainbody; and a light defined in the main body for illuminating the trackingpanel.

In one embodiment, the tracking panel is defined from a translucentmaterial.

In one embodiment, sharing recorded gameplay includes sharing one ormore of an image or a video clip.

In one embodiment, sharing recorded gameplay includes streaming a livevideo feed of a user's gameplay.

Other aspects of the invention will become apparent from the followingdetailed description, taken in conjunction with the accompanyingdrawings, illustrating by way of example the principles of theinvention.

BRIEF DESCRIPTION OF THE DRAWINGS

The invention may best be understood by reference to the followingdescription taken in conjunction with the accompanying drawings inwhich:

FIG. 1 illustrates a perspective view of a controller 10 for interfacingwith an interactive program, in accordance with an embodiment of theinvention.

FIG. 2 illustrates a user's hands holding the controller 10 in thevicinity of a display 30, in accordance with an embodiment of theinvention.

FIG. 3 illustrates combined usage of input devices on the controller 10to control a virtual viewpoint in a three-dimensional virtualenvironment, in accordance with an embodiment of the invention.

FIGS. 4A and 4B illustrate a side angle view and a front view,respectively, of the controller 10, in accordance with an embodiment ofthe invention.

FIG. 5A illustrates a side view of the controller 10, in accordance withan embodiment of the invention.

FIGS. 5B-5M illustrate various tracking features for a tracking panel,in accordance with embodiments of the invention.

FIG. 6 illustrates a perspective view of a controller 10 having aconcave recess 56 defined along a front side of the main body of thecontroller, in accordance with an embodiment of the invention.

FIG. 7 illustrates a bottom view of a controller 10, in accordance withan embodiment of the invention.

FIG. 8 illustrates a cross-section view of the controller 10, showingthe operation of 3D control bar 22.

FIG. 9 illustrates a back perspective view of the controller 10, inaccordance with an embodiment of the invention.

FIG. 10A illustrates another perspective view of the controller 10, inaccordance with an embodiment of the invention.

FIG. 10B illustrates the controller 10 without the 3D control bar, inaccordance with an embodiment of the invention.

FIG. 11A illustrates a top view of a controller device, in accordancewith an embodiment of the invention.

FIG. 11B illustrates a perspective view of a controller device, inaccordance with an embodiment of the invention.

FIG. 12 illustrates hardware and user interfaces that may be used toprovide interactivity with a video game, in accordance with oneembodiment of the present invention.

FIG. 13 illustrates additional hardware that may be used to processinstructions, in accordance with one embodiment of the presentinvention.

FIG. 14 is an exemplary illustration of scene A through scene E withrespective user A through user E interacting with game clients 1102 thatare connected to server processing via the internet, in accordance withone embodiment of the present invention.

FIG. 15 illustrates an embodiment of an Information Service Providerarchitecture.

DETAILED DESCRIPTION

The following embodiments describe methods and apparatus for interfacingwith an interactive program.

It will be obvious, however, to one skilled in the art, that the presentinvention may be practiced without some or all of these specificdetails. In other instances, well known process operations have not beendescribed in detail in order not to unnecessarily obscure the presentinvention.

FIG. 1 illustrates a perspective view of a controller 10 for interfacingwith an interactive program, in accordance with an embodiment of theinvention. The controller 10 includes a main body 12 and extensions 14 aand 14 b. The extension 14 a and 14 b are configured to be held by theuser's left and right hands, respectively, and thus function as handlesor handle portions to enable secure gripping of the controller by theuser. On the top surface of the main body 12, there are included variousinput devices, such as buttons 16, joysticks 18, and directional pad 20.Also shown is the top portion of a 3D control bar 22 which extendsthrough the main body of the controller from top to bottom, and isdescribed in further detail below. A speaker 24 is provided for playingsounds which provide feedback to the user.

Additionally, the controller 10 includes a touch panel 26 defined alongthe back side of the main body which faces the user when the controlleris held in a conventional position. The touch panel 26 is oriented in asubstantially vertical fashion and situated between the extensions 14 aand 14 b so that a user holding the controller by the extensions canreadily use the touch panel with the thumb of either hand. The touchpanel 26 utilizes a touch-sensitive technology (e.g. resistive,capacitive, etc.) to detect touch gestures. In the illustratedembodiment, the touch panel 26 also has a slight outward curvature fromtop to bottom which provides a tactile sensation by virtue of its shapethat enables the user to readily determine the approximate verticalposition of their thumb on the touch panel based on feeling alone.

In another embodiment, the controller 10 may include one or moremicrophones for capturing sound from the interactive environment. Insome embodiments, the microphones may be arranged as an array ofmicrophones. In one embodiment, the arrangement constitutes a lineararray of microphones. When three or more microphones are included in thearray of microphones, it is possible to determine the location of asound source relative to the microphone array based on analysis of thecaptured audio data from the microphone array. More specifically, asound source can be localized relative to the microphone array based onthe relative timing of its sound as captured by each of the microphonesof the microphone array. Taken in combination with the known locationand orientation of the controller (e.g. as determined based on sensorsand tracking methods as defined elsewhere herein), and by extension theknown location and orientation of the microphone array, then thelocation of the sound source within the interactive environment can bedetermined. Furthermore, captured sound can be processed to excludesounds which do not emanate from a certain region of the interactiveenvironment.

In one embodiment, the touch panel 26 can be coupled with a displayscreen to provide a touchscreen interface as part of the controller. Thetouchscreen interface can be controlled by an interactive application todisplay various images in accordance with the interactive application.For example, the touchscreen might display an image which delineatesregions of the touchscreen that correspond to various functionalitiesfor the interactive application. As another example, the touchscreenmight display a line which indicates a gesture that the user mayperform. The user might trace the line shown on the touchscreen so as toperform the indicated gesture. As yet another example, the touchscreenmight be configured to display a gesture or gestures of the user, byproviding a visual trace line of where the user touched and swiped/movedtheir finger(s) along the touchscreen. In one embodiment, thetouchscreen is configured to display the most recent gesture or gestureswhich have been performed on the touchscreen. By way of example, oldergestures may be eliminated from display when a new gesture is detected,or trace lines may be configured to fade from display over time or beeliminated from display after a preset time has elapsed.

Placement of one or more microphones on the controller can beadvantageous over alternative microphone placements (e.g. near adisplay, on a separate device) because the controller is held by theuser and therefore in close proximity to the user. Furthermore, wheremultiple users are present, each operating a controller, then theproximity of each users' corresponding controller to himself/herselfhelps diminish crosstalk, thereby facilitating better identification ofsound as coming from a particular user. Moreover, when multiplecontrollers are utilized, then the captured audio data from the multiplecontrollers can be analyzed in combination, along with the locations andorientations of the controllers, to enable determination of thelocations from which sounds originate with high levels of accuracy.

FIG. 2 illustrates a user's hands holding the controller 10 in thevicinity of a display 30, in accordance with an embodiment of theinvention. As can be seen, the vertical orientation of the touch panel26 is such that the touch panel is substantially parallel to the displayscreen 30 when the user holds the controller in a normal fashion andfaces the display screen. In this manner, the touch panel 26 can beutilized as an intuitive input mechanism for providing directional inputin the plane parallel to the display screen 30 (the x-y plane). Forexample, the touch panel 26 could be utilized to control an on-screencursor or reticle, or to scroll vertically and horizontally, or provideother types of input along the x-y plane.

FIG. 3 illustrates combined usage of input devices on the controller 10to control a virtual viewpoint in a three-dimensional virtualenvironment, in accordance with an embodiment of the invention. Asshown, the controller 10 is operated in the vicinity of a display 30. Insome prior art systems, a controller having two vertically orientedjoysticks is utilized to control the movement and orientation of avirtual viewpoint associated with a character or other object in a videogame. Typically, one joystick will control the x-axis translation (rightand left translation) and z-axis translation (forward and reversetranslation) of the viewpoint, while the other joystick controls thepitch (vertical rotation) and yaw (horizontal rotation) of theviewpoint. However, the use of a vertically oriented joystick to controlpitch and yaw is non-intuitive, especially as the forward/backwardjoystick input must be translated into a pitch movement which isessentially a vertically oriented movement. In fact, it is not uncommonfor systems to offer a “normal” setting wherein forward and backwardmotion of the joystick corresponds to positive and negative changes inpitch, respectively, and the exact opposite configuration (sometimestermed an airplane-style configuration) wherein forward and backwardmotion of the joystick corresponds to negative and positive changes inpitch, respectively.

However, in contrast to the non-intuitive control scheme just described,the touch panel 26 and joystick 18 of the presently disclosed controller10 can be utilized in a much more intuitive fashion to enable control ofa virtual viewpoint. In one embodiment, the joystick 18 is utilized tocontrol x-axis translation and z-axis translation, and the touch panel26 is utilized to control pitch and yaw. As the touch panel 26 isdefined along the side of the main body of the controller and verticallyoriented so as to be substantially parallel to the plane of the display30, its usage to control pitch and yaw is intuitive for the user. In theillustrated embodiment, a right joystick 18 is shown for ease ofdescription only, and not by way of limitation. In embodiments of thecontroller 10 having left and right joysticks, one or both of thejoysticks may be configured to control x-axis and z-axis translation.

In still other embodiments, it will be appreciated that the touch panel26 and joystick 28 can be combined in other ways to provide control of avirtual viewpoint. For example, the touch panel 26 could be utilized tocontrol pitch and x-axis translation, while the joystick 18 is utilizedto control yaw and z-axis translation. Furthermore, while reference hasbeen made to control of a virtual viewpoint, the presently describedcontrol schemes can be applied to control the motion, orientation, orposition of any type of character, vehicle, weapon, or other objectwithin a virtual environment.

FIGS. 4A and 4B illustrate a side angle view and a front view,respectively, of the controller 10, in accordance with an embodiment ofthe invention. As shown, the controller 10 includes a tracking panel 52defined along the front side of the main body opposite the back side onwhich the touch panel is located. The tracking panel 52 is illuminatedand can be visually tracked in accordance with image recognitiontechnologies to determine the location and orientation of the controller10. The tracking panel 52 can be shaped to provide for a wide angle ofvisibility. For example, in the illustrated embodiment, the trackingpanel 52 includes a top upward facing portion 54 a and a bottom downwardfacing portion 54 b which combined provide for a wide angle ofvisibility to enable the controller 10 to be visually tracked in a largevariety of positions. The tracking panel 52 can be defined from atranslucent material and illuminated from within by a light source suchas one or more LEDs.

FIG. 5A illustrates a side view of the controller 10, in accordance withan embodiment of the invention. As can be seen, the tracking panel 52 isdefined so as to also function as a support stand for the front portionof the controller, thus preventing the bottom triggers 50 fromaccidentally being depressed.

It will be appreciated that in various embodiments the tracking panel 52can have any of various tracking features having various shapes, sizes,or forms. Some examples of such shapes and forms are provided withreference to FIGS. 5B-5M. In one embodiment illustrated at FIG. 5B, thetracking panel can define a single rectangular shape. In otherembodiments, the tracking panel can have any other shape, such as acircle, oval, triangle, square, polygon, etc. In another embodiment, thetracking panel may include multiple rectangular shapes, as shown at FIG.5C. It will be appreciated that the multiple rectangular shapes may beilluminated to be the same color or different colors. While eightrectangular shapes are shown in FIG. 5C, there may be any number ofrectangular shapes. In the embodiment of FIG. 5D, the tracking panel isdefined by multiple circular shapes. In the embodiment of FIG. 5DE thetracking panel is defined by a single ovoid shape. In the embodiment ofFIG. 5F the tracking panel includes multiple shapes, including a square,triangle, circle, and an “X,” all of which are horizontally aligned.

It will be appreciated that any combination of shapes may be utilized inembodiments of the invention.

When multiple tracking features (e.g., multiple shapes which may be thesame or different shapes) are utilized, they may be individuallyilluminated to facilitate identification of specific controllers whenmultiple controllers are present. For example, a first controller may becontrolled to illuminate a specific one of or combination of the shapes,a second controller may be controlled to illuminate a different specificone of or combination of the shapes, and so forth. In this manner, eachof the multiple controllers can be identified and distinguished fromeach other based on analysis of captured images of the trackingfeatures, as each controller is configured to illuminate a unique one ofor combination of the shapes which are present on the controller astracking features.

In the embodiment of FIG. 5G the tracking panel includes a plurality ofvertically oriented rectangles which are positioned as a horizontalarray. In the embodiment of FIG. 5H the tracking panel includes multiplehorizontally oriented rectangles arranged in a vertical array. FIG. 5Iillustrates an embodiment wherein the tracking panel includes letters.It will be appreciated that the tracking panel may include any letters,numbers, symbols, or other characters in accordance with variousembodiments of the invention. While embodiments shown have includedmultiple shapes separated from each other, it will be appreciated thatin other embodiments, such as that shown at FIG. 5J, there may bemultiple shapes arranged adjacent to each other. In the illustratedembodiment of FIG. 5J, the tracking panel defines multiple rectanglesarrayed adjacent to each other with no separation between adjacentrectangles.

It will also be appreciated that the tracking panel may be defined tohave various three-dimensional shapes. For example, FIG. 5K illustratesa cross-section view of a tracking panel including a concave-shapedfeature for tracking. FIG. 5L illustrates a cross-section view of atracking panel including a convex-shaped feature for tracking. Theforegoing example of features which may be defined as part of a trackingpanel are provided by way of example only, and not by way of limitation.It will be appreciated by those skilled in the art that in various otherembodiments, the tracking panel may include features of any shape, size,or form.

FIG. 5M illustrates a tracking panel defined by a matrix of pixels, eachof which may be individually illuminated. In the illustrated embodiment,the particular pattern illuminated by the pixels of the tracking panelcan be configured to have any of various designs, and may be configuredto display different patterns for different controllers when multiplecontrollers are in operation.

FIG. 6 illustrates a perspective view of a controller 10 having aconcave recess 56 defined along a front side of the main body of thecontroller, in accordance with an embodiment of the invention. Theconcave recess can be illuminated and utilized for visual tracking ofthe controller. In one embodiment, the concave recess is differentiallylighted: for example, from top to bottom or side to side, employingdifferent colors or different lightness/darkness, with a smooth or sharptransition, or combinations thereof. The differential lighting incombination with changes in size, shape or orientation of the concaverecess in captured images of the controller can be detected and analyzedto determine the position and orientation of the controller relative toan image capture device. In various embodiments, the particular shape ofthe concave recess 56 may vary, both in terms of its surface shape (theshape defined by the concave recess at the surface of the controller)and the shape of the recessed portion. For example, the surface shapemay be a rounded shape as shown, a rectangle or bar, polygon, etc. Andthe recessed portion may be semi-circular, semi-ovoid, angular, faceted,etc. Also, in some embodiments, there may be more than one concaverecess defined on the controller.

FIG. 7 illustrates a bottom view of a controller 10, in accordance withan embodiment of the invention. The bottom portion of the 3D control bar22 is visible protruding from a recess 60.

FIG. 8 illustrates a cross-section view of the controller 10, showingthe operation of the 3D control bar 22. As noted, the 3D control bar 22extends through the main body of the controller from the top surface ofthe main body to its bottom surface. The 3D control bar 22 includes atop pad 70 a and bottom pad 70 b for contacting a user's fingers. In oneembodiment, the control bar is mounted at its central location andpivots about its central location. Thus, horizontal movement of the toppad 70 a causes movement of the bottom pad 70 b in the oppositedirection, and vice versa. Additionally, in one embodiment, the 3Dcontrol bar 22 can be translated in a vertical fashion up or down. Inone embodiment, the 3D control bar 22 can also be translated in ahorizontal direction, moving the entire control bar in a horizontalmanner.

In one embodiment, such freedom of movement of the 3D control bar 22,including pivoting about its central location, and vertical andhorizontal translational movement, is accomplished by way of a floatingmount. For example, in one embodiment, the central portion of thecontrol bar is mounted in a compliant material 72 which allows thecontrol bar to “float” and thereby facilitates such movements. In oneembodiment, the control bar includes a flange 71 to facilitate securemounting in the compliant material 72. The compliant material can be anytype of elastic material which enables the control bar 22 to be moved bya user, but returns the control bar to a normal centered orientationwhen not affected by the user. A sensor 74 detects movement of thecontrol bar 22, and a sensor data analyzer 76 analyzes the raw data fromthe sensor to determine the orientation and/or movement of the controlbar 22.

FIG. 9 illustrates a back perspective view of the controller 10, inaccordance with an embodiment of the invention. FIG. 10A illustratesanother perspective view of the controller 10, in accordance with anembodiment of the invention. FIG. 10B illustrates the controller 10without the 3D control bar, in accordance with an embodiment of theinvention.

It will be appreciated that in various embodiments, a controller mayinclude any of various additional features, including but not limitedto, tactile feedback mechanisms such as a vibration mechanism, variousdata and power connectors such as a USB connector, various inertialsensors such as accelerometers, gyroscopes, and magnetometers, etc.Additional details regarding possible features which may be included ina controller can be found with reference to U.S. application Ser. No.12/259,181, filed Oct. 27, 2008, entitled “Determining Location andMovement of Ball-Attached Controller,” and to U.S. application Ser. No.11/382,036, filed May 6, 2006, entitled “Method and System for ApplyingGearing Effects to Visual Tracking,” the disclosures of which areincorporated by reference herein.

FIG. 11A illustrates a top view of a controller device 100, inaccordance with an embodiment of the invention. The controller 100includes various buttons 102 which can be configured for variouspurposes, as well as directional buttons 104 for providing directionalinput. A left joystick 108 and a right joystick 110 are provided. Asystem button 112 can perform or otherwise provide access to varioussystem functions such as exiting a video game, suspending or turning offa game console, turning off the controller device 100, etc.

A speaker 114 is provided for allowing audio output to occur at thecontroller, which can enrich a user's gameplay by allowing certain audioto be presented through the controller rather than in conjunction withthe rest of the audio output of the video game. For example, ambientsounds from a virtual environment of a video game might be presentedthrough normal audio mechanisms (e.g. as part of the video output of thevideo game) whereas audio from communications (e.g. a phone call, radiocommunication, etc.) are specifically presented through the controllerspeaker 114. When multiple players are engaged in gameplay, audio for aspecific player can be routed to the controller being operated by thatplayer. In this manner, players of a multiplayer game can each receiveaudio that is specific for them, and easily discern that such audio isintended for them, even when engaged in gameplay in the same localgameplay environment.

The controller device 100 includes a touch-sensitive pad 116 tofacilitate touch-based input. An options button 118 can be configured toprovide access to various options, which may be specific to a gameconsole, a cloud gaming platform, a specific video game, or some othercontext. A share button 120 can provide access to a sharing interfacefor sharing a user's gameplay to a social network, such as sharingscreenshots or video clips of the user's gameplay, or initiating livestreaming of the user's active gameplay. In one embodiment, the sharebutton provides access to buffered video of the user's recent gameplay,from which the user may select a portion or a screenshot for sharing toa social network, such as a gaming social network or other socialnetwork.

FIG. 11B illustrates a perspective view of the controller device 100, inaccordance with an embodiment of the invention. As shown, the controller100 includes trigger buttons 130, 132, 134, and 136, which provideadditional inputs for gameplay. A light bar 138 is defined on the frontside of the controller 100 to facilitate identification and tracking ofthe controller 100. The light bar 138 can be illuminated to have aspecific color, and can be identified from captured images of thegameplay environment. It will be appreciated that the location andorientation of the controller 100 can be determined by tracking thelight bar 138.

In one embodiment, a method for storing gameplay is contemplated.Gameplay can be executed by the operating system of a game console inresponse to a user request, which can come in the form of a standardfile operation with respect to a set of data associated with the desiredgameplay. The request can be transmitted from an application associatedwith a game. The gameplay can comprise, for example, video content,audio content and/or static visual content, including wallpapers,themes, code “add-on” content, or any other type of content associatedwith a game. It is contemplated that such content can be user- ordeveloper-generated, free or paid, full or trial, and/or for sale or forrent.

A portion of the gameplay can be buffered, i.e., stored temporarily. Forexample, the previous 15 seconds, the previously completed level, forthe previous action within the gameplay can be stored temporarily, asdescribed further herein. The term “portion” used herein can correspondto any part of the gameplay that is divisible into any related orarbitrary groups of single or multiple bits or bytes of data. Forexample, “portions” of gameplay may correspond to levels, chapters,scenes, acts, characters, backgrounds, textures, courses, actions,songs, themes, durations, sizes, files, parts thereof, and combinationsthereof. Further, portions of gameplay can comprise screenshots orprescribed durations of video capture.

In one embodiment, portions of the gameplay can be stored locally on thegame console in either temporary or permanent storage. Alternatively oradditionally, portions of the gameplay can be transmitted over a networkstored remotely. For example, portions of the gameplay can betransmitted over a wireless or wired network to another computingdevice, to another game console, or to a remote server. Such remoteservers may include social media servers.

Optionally, portions of the gameplay not retrieved from the buffer orportions of the gameplay outside a particular gaming interval (e.g., aparticular duration, level, chapter, course, etc.) can be removed fromthe buffer. This removal process can be completed using standard fileoperations on the operating system.

The portions of the gameplay can be displayed on any of a number ofdisplay devices having access to the stored gameplay. For example, thestored gameplay can be displayed on a television set connected to thegame console from which the gameplay was captured. In another examplethe stored gameplay can be displayed on a computer to which the storedgameplay was transmitted. The stored gameplay can be displayed alone orin conjunction with other information, such as on a social mediawebsite.

In one embodiment, portions of the gameplay are displayed by anothergame console associated with the user other than the user that bufferedor captured the gameplay. According to this embodiment, the portions ofthe gameplay may show a ball being thrown from a first user to a seconduser, from the point of view of the first user. The portions of gameplaycan then be transmitted to the game console of the second user. Thus,the second user can then view the gameplay from the point of view of thefirst user. The second user can also have portions of gameplay storedshowing the ball being thrown by the first user and caught a seconduser, from the point of view of the second user. In this embodiment thesecond user can review the gameplay from both point of view the firstuser and the point of view of the second user. Still further, theportions of the gameplay stored by the second user can be transmitted tothe game console of the first user, so that the first user may reviewthe gameplay from two points of view. This embodiment can apply to anynumber of users having any number of points of view, so the gameplay canbe reviewed from any number of different perspectives.

With respect to storage, transmission and/or display of the portions ofthe gameplay as described herein, it is contemplated that portions ofthe gameplay can be stored, transmitted and displayed as image or videodata. In another embodiment, however, portions of the gameplay can bestored and transmitted as telemetry or metadata representative of theimage or video data, and can be re-created as images or video by a gameconsole or other device prior to display.

In some embodiments, the portion of the gameplay has a predeterminedrelationship with the executed gameplay. For example, the portion of thegameplay can correspond to a certain amount of gameplay prior to thecurrently executing gameplay, such as the previous 10 seconds ofgameplay. In another embodiment, a first portion of the gameplay has apredetermined relationship with a second portion of the gameplay. Forexample, the first portion of the gameplay can correspond to a certainamount of gameplay prior to receipt of a request to capture a secondportion of gameplay, such as the 10 seconds of gameplay prior toselection of a capture button. In each of these embodiments, the amountof gameplay buffered prior to the current gameplay or the requestedgameplay can be configured and adjusted by the user according to his orher particular preferences.

In other embodiments, the buffer is “smart” or “elastic,” such that itcaptures gameplay according to variables without regard to time. In onesuch embodiment, the first portion of the gameplay has a predeterminedrelationship with an event related to the gameplay. For example thefirst portion of the gameplay may be buffered to include a statisticalanomaly, such as a high score being reached, the gathering of a largenumber of points in a short amount of time, the multiple selections ofbuttons on a controller, and other rare events. Such statisticalanomalies can be determined by comparing gameplay metrics to averagemetrics for a particular game or scene or for all games generally. Suchaverage metrics can be stored locally or remotely for comparison. Forexample, the game console can track global high scores for a particulargame, and buffer gameplay in which a user approaches and surpasses thathigh score. In another example, a remote server can track global highscores for a particular game, and can communicate that information tothe game console, which buffers gameplay in which the user approachesand surpasses that high score.

In another example, the portion of the gameplay can be buffered toinclude an achievement, such as a trophy being attained or otherlandmark being reached. Such trophies or landmarks memorialized any goalor gaming achievement, such as a certain number of points beingattained, a certain level being reached, and the like. For example,gameplay can be buffered to include the awarding of a trophy forreaching level 10, for reaching 100,000 points, etc.

Similarly, progress toward reaching an event, in addition to the actualattainment of the trophy or statistical anomaly, can be buffered to beincluded in the portion of the gameplay. For example, a screenshot canbe taken at each of levels one through 10, creating a photo album tomemorialize the receipt of a trophy for reaching level 10. Anotherexample, a video can be taken of the user winning a race for the firstthrough fifth times, where a trophy is awarded for five wins.

Thus, according to embodiments of the invention, at least a portion ofexecuted gameplay can always be kept in a running buffer. In otherwords, when a request to share a portion of the gameplay is received, aportion of the prior gameplay can already be captured to includeprevious footage. For example, if a request to share gameplay isreceived after a user crosses the finish line in a racing game, thebuffered gameplay can include footage of the user crossing the finishline. In other words, a user will be able to capture moments occurringbefore a request is made to share gameplay.

It will be appreciated that the user may share gameplay (e.g. a selectedscreenshot, video, or live gameplay stream) to one or more specificallyselected friends, to their entire social graph, or to any user of thesocial network. The social network can be a gaming social networkassociated with the platform on which the video game is run, or athird-party social network that exists separate from the video game orits platform. The social network can be accessed through an API definedto allow interfacing with the social network. Users to whom gameplay hasbeen shared may receive a notification informing them of the sharedgameplay. Such a notification may take the form of a posting to a socialnews feed, a private message through the social network, an in-gamenotification, an e-mail, a chat notification, etc. Sharing gameplay tothe social network may entail making the gameplay available to othersubsets of users of the social network who may or may not be part of thesharing user's social graph. For example, for a given video game,gameplay may be shared or made available to any user of the socialnetwork who also owns the video game and therefore is granted access toshared gameplay of the video game. Such shared gameplay may be accessedthrough online forums, chat rooms, or other online channels that areavailable only to players of the video game. In one embodiment, a videogame may have a dedicated page or site on the social network. Sharedgameplay can be made available to users accessing the page or site ofthe video game game. Of course, it will be appreciated that from theperspective of the sharing user, options can be provided to allow theuser to specify and tailor who and what forum to which their gameplaywill be shared.

Though various interfaces for sharing can be accessed from a dedicatedbutton press (e.g. pressing the controller share button), it will beappreciated that in other embodiments, some or all of these interfacesmay not be required to facilitate sharing of gameplay to a user socialgraph. For example, in one embodiment, the controller share button canbe configured to capture a screenshot of the user's gameplay whenpressed. The captured screenshot can then be automatically uploaded andshared to the user's social graph.

In another embodiment, pressing the share button on the controllerinitiates recording of gameplay video. When the share button is presseda second time, recording of the gameplay video is stopped, and the videoclip can be uploaded and shared to the user social graph. In oneembodiment, the uploading and sharing of the video clip to the usersocial graph may take place automatically following termination of thevideo recording operation. However, in another embodiment, when theshare button is pressed a second time to stop recording, an interface ispresented to allow the user to customize various options such astrimming the video, selecting a representative screenshot for the video,determine specific users with whom to share the video with, add acaption or title, etc. After customization by the user, the video can beshared with others or otherwise made available for viewing.

In one embodiment, the share button on the controller can be configuredto share a predefined duration of gameplay video on a social network.For example, a user might specify that when the share button is pressed,the previous 10 seconds of gameplay video will be shared to the user'ssocial graph. In another embodiment, it may be specified that when theshare button is pressed, the next 10 seconds of gameplay video will berecorded and shared to the social graph. It should be appreciated thatoptions for trimming the video and performing other types ofcustomization may be applied to the recorded gameplay video.Furthermore, recorded gameplay video of a predefined duration followingthe button trigger can be combined with previously buffered gameplayvideo as has been described.

In yet another embodiment, the share button on the controller device canbe configured to initiate live video streaming of the users activegameplay. A live video streaming can be predefined to be made availableonly to members of the user social graph, or to other smaller or largergroups of users, such as a specific subset of the user social graph, allusers who own or otherwise have access to the same video game, any userof the gaming platform, etc.

FIG. 12 illustrates hardware and user interfaces that may be used toprovide interactivity with a video game, in accordance with oneembodiment of the present invention. FIG. 12 schematically illustratesthe overall system architecture of the Sony® Playstation 3®entertainment device, a console that may be compatible for interfacing acontrol device with a computer program executing at a base computingdevice in accordance with embodiments of the present invention. A systemunit 700 is provided, with various peripheral devices connectable to thesystem unit 700. The system unit 700 comprises: a Cell processor 728; aRambus® dynamic random access memory (XDRAM) unit 726; a RealitySynthesizer graphics unit 730 with a dedicated video random accessmemory (VRAM) unit 732; and an I/O bridge 734. The system unit 700 alsocomprises a Blu Ray® Disk BD-ROM® optical disk reader 740 for readingfrom a disk 740 a and a removable slot-in hard disk drive (HDD) 736,accessible through the I/O bridge 734. Optionally the system unit 700also comprises a memory card reader 738 for reading compact flash memorycards, Memory Stick® memory cards and the like, which is similarlyaccessible through the I/O bridge 734.

The I/O bridge 734 also connects to six Universal Serial Bus (USB) 2.0ports 724; a gigabit Ethernet port 722; an IEEE 802.11b/g wirelessnetwork (Wi-Fi) port 720; and a Bluetooth® wireless link port 718capable of supporting up to seven Bluetooth connections.

In operation, the I/O bridge 734 handles all wireless, USB and Ethernetdata, including data from one or more game controllers 702-703. Forexample when a user is playing a game, the I/O bridge 734 receives datafrom the game controller 702-703 via a Bluetooth link and directs it tothe Cell processor 728, which updates the current state of the gameaccordingly.

The wireless, USB and Ethernet ports also provide connectivity for otherperipheral devices in addition to game controllers 702-703, such as: aremote control 704; a keyboard 706; a mouse 708; a portableentertainment device 710 such as a Sony Playstation Portable®entertainment device; a video camera such as an EyeToy® video camera712; a microphone headset 714; and a microphone 715. Such peripheraldevices may therefore in principle be connected to the system unit 700wirelessly; for example the portable entertainment device 710 maycommunicate via a Wi-Fi ad-hoc connection, whilst the microphone headset714 may communicate via a Bluetooth link.

The provision of these interfaces means that the Playstation 3 device isalso potentially compatible with other peripheral devices such asdigital video recorders (DVRs), set-top boxes, digital cameras, portablemedia players, Voice over IP telephones, mobile telephones, printers andscanners.

In addition, a legacy memory card reader 716 may be connected to thesystem unit via a USB port 724, enabling the reading of memory cards 748of the kind used by the Playstation® or Playstation 2® devices.

The game controllers 702-703 are operable to communicate wirelessly withthe system unit 700 via the Bluetooth link, or to be connected to a USBport, thereby also providing power by which to charge the battery of thegame controllers 702-703. Game controllers 702-703 can also includememory, a processor, a memory card reader, permanent memory such asflash memory, light emitters such as an illuminated spherical section,LEDs, or infrared lights, microphone and speaker for ultrasoundcommunications, an acoustic chamber, a digital camera, an internalclock, a recognizable shape such as the spherical section facing thegame console, and wireless communications using protocols such asBluetooth®, WiFi™, etc.

Game controller 702 is a controller designed to be used with two hands,and game controller 703 is a single-hand controller with an attachment.In addition to one or more analog joysticks and conventional controlbuttons, the game controller is susceptible to three-dimensionallocation determination. Consequently gestures and movements by the userof the game controller may be translated as inputs to a game in additionto or instead of conventional button or joystick commands Optionally,other wirelessly enabled peripheral devices such as the Playstation™Portable device may be used as a controller. In the case of thePlaystation™ Portable device, additional game or control information(for example, control instructions or number of lives) may be providedon the screen of the device. Other alternative or supplementary controldevices may also be used, such as a dance mat (not shown), a light gun(not shown), a steering wheel and pedals (not shown) or bespokecontrollers, such as a single or several large buttons for arapid-response quiz game (also not shown).

The remote control 704 is also operable to communicate wirelessly withthe system unit 700 via a Bluetooth link. The remote control 704comprises controls suitable for the operation of the Blu Ray™ DiskBD-ROM reader 540 and for the navigation of disk content.

The Blu Ray™ Disk BD-ROM reader 740 is operable to read CD-ROMscompatible with the Playstation and PlayStation 2 devices, in additionto conventional pre-recorded and recordable CDs, and so-called SuperAudio CDs. The reader 740 is also operable to read DVD-ROMs compatiblewith the Playstation 2 and PlayStation 3 devices, in addition toconventional pre-recorded and recordable DVDs. The reader 740 is furtheroperable to read BD-ROMs compatible with the Playstation 3 device, aswell as conventional pre-recorded and recordable Blu-Ray Disks.

The system unit 700 is operable to supply audio and video, eithergenerated or decoded by the Playstation 3 device via the RealitySynthesizer graphics unit 730, through audio and video connectors to adisplay and sound output device 742 such as a monitor or television sethaving a display 744 and one or more loudspeakers 746. The audioconnectors 750 may include conventional analogue and digital outputswhilst the video connectors 752 may variously include component video,S-video, composite video and one or more High Definition MultimediaInterface (HDMI) outputs. Consequently, video output may be in formatssuch as PAL or NTSC, or in 720p, 1080i or 1080p high definition.

Audio processing (generation, decoding and so on) is performed by theCell processor 728. The Playstation 3 device's operating system supportsDolby® 5.1 surround sound, Dolby® Theatre Surround (DTS), and thedecoding of 7.1 surround sound from Blu-Ray® disks.

In the present embodiment, the video camera 712 comprises a singlecharge coupled device (CCD), an LED indicator, and hardware-basedreal-time data compression and encoding apparatus so that compressedvideo data may be transmitted in an appropriate format such as anintra-image based MPEG (motion picture expert group) standard fordecoding by the system unit 700. The camera LED indicator is arranged toilluminate in response to appropriate control data from the system unit700, for example to signify adverse lighting conditions. Embodiments ofthe video camera 712 may variously connect to the system unit 700 via aUSB, Bluetooth or Wi-Fi communication port. Embodiments of the videocamera may include one or more associated microphones and also becapable of transmitting audio data. In embodiments of the video camera,the CCD may have a resolution suitable for high-definition videocapture. In use, images captured by the video camera may for example beincorporated within a game or interpreted as game control inputs. Inanother embodiment the camera is an infrared camera suitable fordetecting infrared light.

In general, in order for successful data communication to occur with aperipheral device such as a video camera or remote control via one ofthe communication ports of the system unit 700, an appropriate piece ofsoftware such as a device driver should be provided. Device drivertechnology is well-known and will not be described in detail here,except to say that the skilled man will be aware that a device driver orsimilar software interface may be required in the present embodimentdescribed.

FIG. 13 illustrates additional hardware that may be used to processinstructions, in accordance with one embodiment of the presentinvention. Cell processor 728 has an architecture comprising four basiccomponents: external input and output structures comprising a memorycontroller 860 and a dual bus interface controller 870A, B; a mainprocessor referred to as the Power Processing Element 850; eightco-processors referred to as Synergistic Processing Elements (SPEs)810A-H; and a circular data bus connecting the above components referredto as the Element Interconnect Bus 880. The total floating pointperformance of the Cell processor is 218 GFLOPS, compared with the 6.2GFLOPs of the Playstation 2 device's Emotion Engine.

The Power Processing Element (PPE) 850 is based upon a two-waysimultaneous multithreading Power 570 compliant PowerPC core (PPU) 855running with an internal clock of 3.2 GHz. It comprises a 512 kB level 2(L2) cache and a 32 kB level 1 (L1) cache. The PPE 850 is capable ofeight single position operations per clock cycle, translating to 25.6GFLOPs at 3.2 GHz. The primary role of the PPE 850 is to act as acontroller for the Synergistic Processing Elements 810A-H, which handlemost of the computational workload. In operation the PPE 850 maintains ajob queue, scheduling jobs for the Synergistic Processing Elements810A-H and monitoring their progress. Consequently each SynergisticProcessing Element 810A-H runs a kernel whose role is to fetch a job,execute it and synchronized with the PPE 850.

Each Synergistic Processing Element (SPE) 810A-H comprises a respectiveSynergistic Processing Unit (SPU) 820A-H, and a respective Memory FlowController (MFC) 840A-H comprising in turn a respective Dynamic MemoryAccess Controller (DMAC) 842A-H, a respective Memory Management Unit(MMU) 844A-H and a bus interface (not shown). Each SPU 820A-H is a RISCprocessor clocked at 3.2 GHz and comprising 256 kB local RAM 830A-H,expandable in principle to 4 GB. Each SPE gives a theoretical 25.6GFLOPS of single precision performance. An SPU can operate on 4 singleprecision floating point members, 4 32-bit numbers, 8 16-bit integers,or 16 8-bit integers in a single clock cycle. In the same clock cycle itcan also perform a memory operation. The SPU 820A-H does not directlyaccess the system memory XDRAM 726; the 64-bit addresses formed by theSPU 820A-H are passed to the MFC 840A-H which instructs its DMAcontroller 842A-H to access memory via the Element Interconnect Bus 880and the memory controller 860.

The Element Interconnect Bus (EIB) 880 is a logically circularcommunication bus internal to the Cell processor 728 which connects theabove processor elements, namely the PPE 850, the memory controller 860,the dual bus interface 870A,B and the 8 SPEs 810A-H, totaling 12participants. Participants can simultaneously read and write to the busat a rate of 8 bytes per clock cycle. As noted previously, each SPE810A-H comprises a DMAC 842A-H for scheduling longer read or writesequences. The EIB comprises four channels, two each in clockwise andanti-clockwise directions. Consequently for twelve participants, thelongest step-wise data-flow between any two participants is six steps inthe appropriate direction. The theoretical peak instantaneous EIBbandwidth for 12 slots is therefore 96B per clock, in the event of fullutilization through arbitration between participants. This equates to atheoretical peak bandwidth of 307.2 GB/s (gigabytes per second) at aclock rate of 3.2 GHz.

The memory controller 860 comprises an XDRAM interface 862, developed byRambus Incorporated. The memory controller interfaces with the RambusXDRAM 726 with a theoretical peak bandwidth of 25.6 GB/s.

The dual bus interface 870A,B comprises a Rambus FlexIO® systeminterface 872A,B. The interface is organized into 12 channels each being8 bits wide, with five paths being inbound and seven outbound. Thisprovides a theoretical peak bandwidth of 62.4 GB/s (36.4 GB/s outbound,26 GB/s inbound) between the Cell processor and the I/O Bridge 734 viacontroller 870A and the Reality Simulator graphics unit 730 viacontroller 870B.

Data sent by the Cell processor 728 to the Reality Simulator graphicsunit 730 will typically comprise display lists, being a sequence ofcommands to draw vertices, apply textures to polygons, specify lightingconditions, and so on.

FIG. 14 is an exemplary illustration of scene A through scene E withrespective user A through user E interacting with game clients 1102 thatare connected to server processing via the internet, in accordance withone embodiment of the present invention. A game client is a device thatallows users to connect to server applications and processing via theinternet. The game client allows users to access and playback onlineentertainment content such as but not limited to games, movies, musicand photos. Additionally, the game client can provide access to onlinecommunications applications such as VOIP, text chat protocols, andemail.

A user interacts with the game client via controller. In someembodiments the controller is a game client specific controller while inother embodiments, the controller can be a keyboard and mousecombination. In one embodiment, the game client is a standalone devicecapable of outputting audio and video signals to create a multimediaenvironment through a monitor/television and associated audio equipment.For example, the game client can be, but is not limited to a thinclient, an internal PCI-express card, an external PCI-express device, anExpressCard device, an internal, external, or wireless USB device, or aFirewire device, etc. In other embodiments, the game client isintegrated with a television or other multimedia device such as a DVR,Blu-Ray player, DVD player or multi-channel receiver.

Within scene A of FIG. 14, user A interacts with a client applicationdisplayed on a monitor 1104A using a controller 1106A paired with gameclient 1102A. Similarly, within scene B, user B interacts with anotherclient application that is displayed on monitor 1104B using a controller1106B paired with game client 1102B. Scene C illustrates a view frombehind user C as he looks at a monitor displaying a game and buddy listfrom the game client 1102C. While FIG. 14 shows a single serverprocessing module, in one embodiment, there are multiple serverprocessing modules throughout the world. Each server processing moduleincludes sub-modules for user session control, sharing/communicationlogic, user geo-location, and load balance processing service.Furthermore, a server processing module includes network processing anddistributed storage.

When a game client 1102 connects to a server processing module, usersession control may be used to authenticate the user. An authenticateduser can have associated virtualized distributed storage and virtualizednetwork processing. Examples items that can be stored as part of auser's virtualized distributed storage include purchased media such as,but not limited to games, videos and music etc. Additionally,distributed storage can be used to save game status for multiple games,customized settings for individual games, and general settings for thegame client. In one embodiment, the user geo-location module of theserver processing is used to determine the geographic location of a userand their respective game client. The user's geographic location can beused by both the sharing/communication logic and the load balanceprocessing service to optimize performance based on geographic locationand processing demands of multiple server processing modules.Virtualizing either or both network processing and network storage wouldallow processing tasks from game clients to be dynamically shifted tounderutilized server processing module(s). Thus, load balancing can beused to minimize latency associated with both recall from storage andwith data transmission between server processing modules and gameclients.

The server processing module has instances of server application A andserver application B. The server processing module is able to supportmultiple server applications as indicated by server application X₁ andserver application X₂. In one embodiment, server processing is based oncluster computing architecture that allows multiple processors within acluster to process server applications. In another embodiment, adifferent type of multi-computer processing scheme is applied to processthe server applications. This allows the server processing to be scaledin order to accommodate a larger number of game clients executingmultiple client applications and corresponding server applications.Alternatively, server processing can be scaled to accommodate increasedcomputing demands necessitated by more demanding graphics processing orgame, video compression, or application complexity. In one embodiment,the server processing module performs the majority of the processing viathe server application. This allows relatively expensive components suchas graphics processors, RAM, and general processors to be centrallylocated and reduces to the cost of the game client. Processed serverapplication data is sent back to the corresponding game client via theinternet to be displayed on a monitor.

Scene C illustrates an exemplary application that can be executed by thegame client and server processing module. For example, in one embodimentgame client 1102C allows user C to create and view a buddy list 1120that includes user A, user B, user D and user E. As shown, in scene C,user C is able to see either real time images or avatars of therespective user on monitor 1104C. Server processing executes therespective applications of game client 1102C and with the respectivegame clients 1102 of users A, user B, user D and user E. Because theserver processing is aware of the applications being executed by gameclient B, the buddy list for user A can indicate which game user B isplaying. Further still, in one embodiment, user A can view actual ingame video directly from user B. This is enabled by merely sendingprocessed server application data for user B to game client A inaddition to game client B.

In addition to being able to view video from buddies, the communicationapplication can allow real-time communications between buddies. Asapplied to the previous example, this allows user A to provideencouragement or hints while watching real-time video of user B. In oneembodiment two-way real time voice communication is established througha client/server application. In another embodiment, a client/serverapplication enables text chat. In still another embodiment, aclient/server application converts speech to text for display on abuddy's screen.

Scene D and scene E illustrate respective user D and user E interactingwith game consoles 1110D and 1110E respectively. Each game console 1110Dand 1110E are connected to the server processing module and illustrate anetwork where the server processing modules coordinates game play forboth game consoles and game clients.

FIG. 15 illustrates an embodiment of an Information Service Providerarchitecture. Information Service Providers (ISP) 1370 delivers amultitude of information services to users 1382 geographically dispersedand connected via network 1386. An ISP can deliver just one type ofservice, such as stock price updates, or a variety of services such asbroadcast media, news, sports, gaming, etc. Additionally, the servicesoffered by each ISP are dynamic, that is, services can be added or takenaway at any point in time. Thus, the ISP providing a particular type ofservice to a particular individual can change over time. For example, auser may be served by an ISP in near proximity to the user while theuser is in her home town, and the user may be served by a different ISPwhen the user travels to a different city. The home-town ISP willtransfer the required information and data to the new ISP, such that theuser information “follows” the user to the new city making the datacloser to the user and easier to access. In another embodiment, amaster-server relationship may be established between a master ISP,which manages the information for the user, and a server ISP thatinterfaces directly with the user under control from the master ISP. Inother embodiment, the data is transferred from one ISP to another ISP asthe client moves around the world to make the ISP in better position toservice the user be the one that delivers these services.

ISP 1370 includes Application Service Provider (ASP) 1372, whichprovides computer-based services to customers over a network. Softwareoffered using an ASP model is also sometimes called on-demand softwareor software as a service (SaaS). A simple form of providing access to aparticular application program (such as customer relationshipmanagement) is by using a standard protocol such as HTTP. Theapplication software resides on the vendor's system and is accessed byusers through a web browser using HTML, by special purpose clientsoftware provided by the vendor, or other remote interface such as athin client.

Services delivered over a wide geographical area often use cloudcomputing. Cloud computing is a style of computing in which dynamicallyscalable and often virtualized resources are provided as a service overthe Internet. Users do not need to be an expert in the technologyinfrastructure in the “cloud” that supports them. Cloud computing can bedivided in different services, such as Infrastructure as a Service(IaaS), Platform as a Service (PaaS), and Software as a Service (SaaS).Cloud computing services often provide common business applicationsonline that are accessed from a web browser, while the software and dataare stored on the servers. The term cloud is used as a metaphor for theInternet, based on how the Internet is depicted in computer networkdiagrams and is an abstraction for the complex infrastructure itconceals.

Further, ISP 1370 includes a Game Processing Server (GPS) 1374 which isused by game clients to play single and multiplayer video games. Mostvideo games played over the Internet operate via a connection to a gameserver. Typically, games use a dedicated server application thatcollects data from players and distributes it to other players. This ismore efficient and effective than a peer-to-peer arrangement, but itrequires a separate server to host the server application. In anotherembodiment, the GPS establishes communication between the players andtheir respective game-playing devices exchange information withoutrelying on the centralized GPS.

Dedicated GPSs are servers which run independently of the client. Suchservers are usually run on dedicated hardware located in data centers,providing more bandwidth and dedicated processing power. Dedicatedservers are the preferred method of hosting game servers for mostPC-based multiplayer games. Massively multiplayer online games run ondedicated servers usually hosted by the software company that owns thegame title, allowing them to control and update content.

Broadcast Processing Server (BPS) 1376 distributes audio or videosignals to an audience. Broadcasting to a very narrow range of audienceis sometimes called narrowcasting. The final leg of broadcastdistribution is how the signal gets to the listener or viewer, and itmay come over the air as with a radio station or TV station to anantenna and receiver, or may come through cable TV or cable radio (or“wireless cable”) via the station or directly from a network. TheInternet may also bring either radio or TV to the recipient, especiallywith multicasting allowing the signal and bandwidth to be shared.Historically, broadcasts have been delimited by a geographic region,such as national broadcasts or regional broadcast. However, with theproliferation of fast internet, broadcasts are not defined bygeographies as the content can reach almost any country in the world.

Storage Service Provider (SSP) 1378 provides computer storage space andrelated management services. SSPs also offer periodic backup andarchiving. By offering storage as a service, users can order morestorage as required. Another major advantage is that SSPs include backupservices and users will not lose all their data if their computers' harddrives fail. Further, a plurality of SSPs can have total or partialcopies of the user data, allowing users to access data in an efficientway independently of where the user is located or the device being usedto access the data. For example, a user can access personal files in thehome computer, as well as in a mobile phone while the user is on themove.

Communications Provider 380 provides connectivity to the users. One kindof Communications Provider is an Internet Service Provider (ISP) whichoffers access to the Internet. The ISP connects its customers using adata transmission technology appropriate for delivering InternetProtocol datagrams, such as dial-up, DSL, cable modem, wireless ordedicated high-speed interconnects. The Communications Provider can alsoprovide messaging services, such as e-mail, instant messaging, and SMStexting. Another type of Communications Provider is the Network Serviceprovider (NSP) which sells bandwidth or network access by providingdirect backbone access to the Internet. Network service providers mayconsist of telecommunications companies, data carriers, wirelesscommunications providers, Internet service providers, cable televisionoperators offering high-speed Internet access, etc.

Data Exchange 1388 interconnects the several modules inside ISP 1370 andconnects these modules to users 1382 via network 1386. Data Exchange1388 can cover a small area where all the modules of ISP 1370 are inclose proximity, or can cover a large geographic area when the differentmodules are geographically dispersed. For example, Data Exchange 1388can include a fast Gigabit Ethernet (or faster) within a cabinet of adata center, or an intercontinental virtual area network (VLAN).

Users 1382 access the remote services with client device 1384, whichincludes at least a CPU, a display and I/O. The client device can be aPC, a mobile phone, a netbook, a PDA, etc. In one embodiment, ISP 1370recognizes the type of device used by the client and adjusts thecommunication method employed. In other cases, client devices use astandard communications method, such as html, to access ISP 1370.

Embodiments of the present invention may be practiced with variouscomputer system configurations including hand-held devices,microprocessor systems, microprocessor-based or programmable consumerelectronics, minicomputers, mainframe computers and the like. Theinvention can also be practiced in distributed computing environmentswhere tasks are performed by remote processing devices that are linkedthrough a wire-based or wireless network.

With the above embodiments in mind, it should be understood that theinvention can employ various computer-implemented operations involvingdata stored in computer systems. These operations are those requiringphysical manipulation of physical quantities. Any of the operationsdescribed herein that form part of the invention are useful machineoperations. The invention also relates to a device or an apparatus forperforming these operations. The apparatus can be specially constructedfor the required purpose, or the apparatus can be a general-purposecomputer selectively activated or configured by a computer programstored in the computer. In particular, various general-purpose machinescan be used with computer programs written in accordance with theteachings herein, or it may be more convenient to construct a morespecialized apparatus to perform the required operations.

The invention can also be embodied as computer readable code on acomputer readable medium. The computer readable medium is any datastorage device that can store data, which can be thereafter be read by acomputer system. Examples of the computer readable medium include harddrives, network attached storage (NAS), read-only memory, random-accessmemory, CD-ROMs, CD-Rs, CD-RWs, magnetic tapes and other optical andnon-optical data storage devices. The computer readable medium caninclude computer readable tangible medium distributed over anetwork-coupled computer system so that the computer readable code isstored and executed in a distributed fashion.

Although the method operations were described in a specific order, itshould be understood that other housekeeping operations may be performedin between operations, or operations may be adjusted so that they occurat slightly different times, or may be distributed in a system whichallows the occurrence of the processing operations at various intervalsassociated with the processing, as long as the processing of the overlayoperations are performed in the desired way.

Although the foregoing invention has been described in some detail forpurposes of clarity of understanding, it will be apparent that certainchanges and modifications can be practiced within the scope of theappended claims. Accordingly, the present embodiments are to beconsidered as illustrative and not restrictive, and the invention is notto be limited to the details given herein, but may be modified withinthe scope and equivalents of the appended claims.

What is claimed is:
 1. A controller for interfacing wirelessly with acomputing device, comprising: a housing defined by a main body, a firstextension extending from a first end of the main body, and a secondextension extending from a second end of the main body, the firstextension and the second extension for holding by a first hand and asecond hand of a user, respectively; a touchscreen defined along the topsurface of the main body between the first extension and the secondextension; a first set of buttons disposed on the top surface of themain body proximate to the first extension and on a first side of thetouchscreen; and a second set of buttons disposed on the top surface ofthe main body proximate to the second extension and on a second side ofthe touchscreen.
 2. The controller of claim 1, wherein the touchscreenis configured to display images controlled by an interactive applicationthat is executed by the computing device.
 3. the controller of claim 1,further comprising: a translucent panel defined along a front side ofthe main body; and a light defined in the main body for illuminating thetranslucent panel; wherein the translucent panel is configured to bevisually tracked to determine a location and/or an orientation of thecontroller.
 4. The controller of claim 1, further comprising: a buttonpositioned along the top surface of the main body, the button beingconfigured to activate a sharing interface for sharing gameplay to asocial graph.
 5. The controller of claim 4, wherein sharing gameplayincludes sharing one or more of an image, a pre-recorded video, or alive video feed of a user's gameplay.
 6. The controller of claim 4,wherein the shared gameplay is defined by a portion of buffered video ofthe user's gameplay.
 7. A controller for interfacing with a gameconsole, comprising: a housing defined by a main body, a first extensionextending from a first end of the main body, and a second extensionextending from a second end of the main body, the first extension andthe second extension for holding by a first hand and a second hand of auser, respectively; a wireless communication device for processingcommunication to and from the game console; an inertial sensor, theinertial sensor configured for detecting motion data of the controller;a touchscreen interface defined along the top surface of the main bodybetween the first extension and the second extension, the touchscreeninterface configured to display images from an interactive applicationexecuted by the game console; a first set of buttons disposed on the topsurface of the main body proximate to the first extension; and a secondset of buttons disposed on the top surface of the main body proximate tothe second extension.
 8. The controller of claim 7, further comprising:a light bar defined along a front side of the main body, wherein thelight bar is configured to be visually tracked to determine a locationand/or an orientation of the controller.
 9. The controller of claim 7,further comprising: one or more joysticks positioned along the topsurface of the main body.
 10. The controller of claim 7, furthercomprising a speaker positioned along the top surface of the main body.11. The controller of claim 7, wherein the wireless communication deviceis configured to send input data from the controller to the gameconsole, and further configured to receive feedback data from the gameconsole.
 12. A controller for interfacing with a game console,comprising: a housing defined by a main body, a first extensionextending from a first end of the main body, and a second extensionextending from a second end of the main body, the first extension andthe second extension for holding by a first hand and a second hand of auser, respectively; a wireless communication device for processingcommunication to and from the game console; at least one inertial sensorselected from one of an accelerometer, a gyroscope or a magnetometer; atouchscreen defined along the top surface of the main body between thefirst extension and the second extension; a first set of buttonsdisposed on the top surface of the main body proximate to the firstextension; and a second set of buttons disposed on the top surface ofthe main body proximate to the second extension.
 13. The controller ofclaim 12, further comprising: a light defined along a front side of themain body, wherein the light is configured to be visually tracked todetermine a location and/or an orientation of the controller.
 14. Thecontroller of claim 12, wherein the wireless communication device isconfigured to send input data from the controller to the game console,and further configured to receive feedback data from the game console.15. The controller of claim 12, wherein the wireless communicationdevice provides for data communication in accordance with a Bluetooth®protocol.
 16. The controller of claim 12, wherein each of the firsthandle and the second handle extend from a back side of the main body.17. The controller of claim 12, further comprising: one or morejoysticks positioned along the top surface of the main body.
 18. Thecontroller of claim 12, further comprising a speaker positioned alongthe top surface of the main body, wherein the speaker is configured topresent a first audio portion from the game console, the first audioportion being separate from a second audio portion that is transmittedby the game console to a display device.